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ABSTRACT 

The evaluation of the 1970-71 bilingual education 
program in northern Lubbock, Texas, is provided in this paper. The 
main purpose of the program was to plan, develop, operate, evaluate, 
and, if necessary, modify a bilingual program which could be operated 
in self-contained classrooms. The 4 major components of the program, 
which included 5 kindergarten and 5 Ist-grade sections located at 5 
elementary schools, were instruction, staff development, materials 
development, and community involvement. In evaluating the program, a 
random sample selected from students from the total program was used. 
English language activities, the language used by the children in 
unsupervised situations, and the Test of Basic Experiences 
administered in both English and Spanish were used to measure 
language growth. The measures employed indicated that the program’s 
objectives, based upon achievement of educational objectives rather 
than upon comparison of rate of gain, were being approached. (NQ) 
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EVALUATION REPORT 
Bilingual Education Progr 
Lubbock, Texas 




This bilingual education program began with a pre-school program 



in the L969-70 school year and expanded to include first grade in 



1970-71* The setting for the program was the northern crescent of the 
150,000 population of Lubbock, Texas; that section contains a higher 
incidence of Mexican American pupils than does the remainder of the 



city. 

The 1970-71 program utilised five elementary schools as program 
sites. Each site reflects some unique demographic characteristics, 
but, taken to total, they seem to represent a good cross-section of 
the Nexican American population and includes a fairly largo mixture 
of other ethnic groups. 

The classes included five sections of kindergarten and five at 
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first grade level. One site, Parkway, was added because of the re- 
location of a large, number of residents from the Guadalupe barrio 
after a tornado in May, 1970* Because of these and other more normal 
population shifts, the first grade classes included many children 
who had not had bilingual experience in kindergarten. 

The program was developed with four major components composed 
of instruction, staff development, materials development, and com- 
munity involvement, and was devoted to the planning, development, 
operation, evaluation and modification of a program kdiich could be 
operated to self-contained classrooms with bilingual (English/Spani3h) 
instructional, capability. 

FILMED FROM BEST AVAILABLE COPY 
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This review is based largely upon findings and perceptions of the 
evaluator* His activities and data collection have been guided by staff, 
program manager and auditor comments and reports* It is aimed at re- 
viewing the evaluation design, which, though inclusive of a number of ob- 
jectives, does not purport to measure or describe all the cognitive, nor 
certainly all the affective impact of the program* 

* During the progress of the program, the evaluator, partially because 
of comments by the auditor, began to collect case study type data on a 
random selection of pupils who had been in the program for both years. 

This sociological approach may, over the tern of the project, provide 
data to complement the more routine educational measurements reported* 

Instruction 

While specified instructional activities were conducted in Spanish, 
it was also used incidentally for instructional assistance in other 
teaching and informal situations* Staff estimates, while almost im- . 
possible to verify, of their use of language were taken separately for 
teachers and aides. These estimates were taken by the evaluator during 
staff meetings with each staff member being asked to calculate: 

What percent of the time did .you spend today in instruction in Spanish? 

Pre Mid-Year End 

Aides M-45$ R=25-*0$ M=52$ R=15-75# M=40$ R^=15-70$ 

Teachers N«52$ R«=25-75$ M=36$ R=0 -85$ M=33$ R=10-75$ 

These self reports plus independent observations indicate that attempts 
were being made to maintain the suggested amounts of Spanish* 

English language activities had as a major component the development 
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of oral language* To this end^ much interest center, small group, and 
individual help activities were used* Measures of language growth were 
considered to be central to achievement of objectives* 

One of the product objectives related to language indicated that: 



One year 


•’s growth in mental age 


will be reflected from the Peabody 


Picture 


Vocabulary pre and post 


tests* 




Grade 


Language 


Whole Group Means 






Pre (Sept* -Oct * ) 


Post (April) 
based on seven 
months instruction 


K 


Spanish 


2-7 


4-6 


K 


English 


2 -10 


4 - 5 


1 


Spanish 


3 -11 


4-7 


1 


English 


4-4 


4-9 



From the foregoing, which is the result of a random sample, the 
growth is shown to favor the Spanish by about one month of the almost 
two years growth recorded for Kindergarten* A lesser amount of growth 
in Grade 1 could be allocated to many factors, including newness of 
program and newness of teachers in the program* 

Some comparative data is available from testing done last year at 
the kindergarten level* There are at least three major complicating 
factors even though some data analysis may be made. The factors include: 
inability to equate kindergarten groups for the two yeai*3, the unrelia- 
bility of testing of young children, and the differences in examiner 
(during the first year teacher and aides administered tests, while this 
year one person did the language testing)* With all of these qualifi- 
cations, it still seems to be a logical assumption that the two groups of 
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kindergarten pupils (1969 and 1970) are quite similar since they were 
from the same population area generally, arri there was little age dif- 
ference. The two years’ scores are graphically portrayed: 

PEABODY PICTURE VOCABULARY TEST 
(in M. A. means) 



GRADE K 



ENGLISH 





1969-70 


0 12 


3 4 5 6 7 


1969-70 


u 


Entry 




(3-0) 


Entry 




Spring 


77771 


'/ (3-10) 


Spring 


— *\ 
I 

-J 


1970-71 


0 12 


3 4 5 6 7 


1970-71 


! 

-4 


Entry 


/'/'/'/ 


(2-10) 


Entry 




Spring 


//, l f l / / /> 


W/l (4-5) 


Spring 



SPANISH 
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5 6 7 
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Prom the test results, and perhaps from more stringent administra- 
tion of testing, it does appear that language usage at entry was slightly 
lower for this year’s kindergarten group. 

GRADE 1 



1970-71 

Entry 

Spring 



ENGLISH 

0123456 ? 



TTTTT71 

fulfil (4-4) 

T v m r n 

fjjl LLLL (±21 



1970-71 

Entry 



SPANISH 

01 2 3 4 567 



miujf 


( 3 - 11 ) 


wttv, 


( 4 - 7 ) 



No comparative data can be reported for Grade 1 since this is the 
first year for that level. A representative testing indicates the 
above. 
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A major difference in evaluation this year was to randomly select 
pupils from the total program to show a representative sample rather 
than to identify rooms which necessitated some further explanation about 
the compsition of the section* Sampling is always subject to some 
question, but the use of a table of random numbers and adherence to those 
random assignments allows the results to be presented as representative 
of the total population without bias. 

Another measure of language is the language used by children in 
unsupervised Situations. This was initiated to assist curriculum de- 
velopment and in pacing of both languages. One of the measures taken 
was through estimation by teachers. Since the estimate for entry was 
taken after the program was well underway, it is probably that estimates 
could not be reliable. Nevertheless, the following" chart shows how 
teachers estimate change for the first half year. 

There was another chock of the above taken by the testing specialist 
of pupils in unsupervised play. Using the device of time sampling, 
children were watched for limited periods .in the middle of the year, and 
a tally made for the language used (English or Spanish) as they inter- 
acted with other children. Some sensitivity was noted in that children 
who otherwise responded a majority of the time in Spanish would frequently 
respond to or initiate speech to monolingual English speakers (e.g. a 
Negro boy who was in the play group) in English. There are differences 
in these findings and that estimated by teachers. The reasons for dif- 
ferences include differences in measurement techniques and also because 
the sensitivity noted above might have been involved in causing youngsters 
to show greater response in school situations in English. From the 

O * Charfc is Page following 
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BI LINGUAL ELEMENTARY EDUCATION PROGRAM 
Lubbock Public Schools 

.LANGUAGE SURVEY PERCENTAGES 

1970-71 



GRADE 


K 


K 


1 


1 


TOTALS 


DATE 


Sept. 

1970 


Jan. 

1971 


Sept. 

1970 


Jan. 

1971 


Sept . 
1970 


Jan. 

1971. 


ENROLLMENT 


99 


103 


113 


114 


212 


217 


. ^'Percent /homes /Spanish dominant 


79.80 




72.57 




75.94 




. Percent/homes/English dominant 


20.20 




18.58 




19.34 




. Percent/understanding Spanish 


92.93 


98.06 


S9.38 


97.37 


91.04 


97.70 


. Percent/understanding English 


75.76 


98.06 


91.15 


96.49 


83.96 


97.24 


. Percent/speak Spanish only 


35.35 


7.77 


12.39 


5.26 


23.11 


6.45 


. Perceot/speak English only 


14.14 


3.88 


7.08 


5.26 


10.38 


4.*L 

V 


. Percent/speak Spanish & English 


50.51 


87.38 


77.00 


89.47 


64.62 


88.48 


. Percent/speak mostly Spanish as 
they interact with peers in 
non-teacher directed activi- 
ties (play groups, break, etc.) 


55.56 


38.83 


38.94 


26.32 


46.70 


32.26 


. Percent/speak mostly English as 
they interact with peers in 
non-teacher directieo activi- 
ties(play groups, break, etc.) 


44 . 44 


58.25 


55.75 


63.16 


50.47 


60.83 



* Percent in each instance is percent of children enrolled 
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observation the following were noted: 



Level 



K 

1 



Number of Children 
Sampled 



20 

24 



Unsupervised Language Choice 
(Average responses per child) 
English Spanish 

3 per minute 4*4 per minute 

3*2 per minute 2*0 per minute 



Another instrument used for instructional measures was the Test of 
Basic Experiences* Using sampling techniques, a small group was tested 
in each language* Although only a brief period has elapsed, the data as 
reported, in standard scores, means: 

The conversion to standard scores indicates a mean of 50 which allows for 
better comparison than do raw or percentile scores* 



TEST OF BASIC EXPERIENCES 



Level K 



Language 



Pre-Test 

Spanish 39 (R 27-47) 
English 40 (R 25-56) 



Post-Test 

Spanish 46 (R 36-51) 
English 42 (R 29-64) 



Math Spanish 39 (R 27-46) 

English 45 (R 29-61 ) 



Spanish 47 (R 38-56) 
English 54 (R 47-69) 



The TOBE instrument, according to the technical manual, is a 
series of standardized group tests which provide an indication of how 
well a child’s experiences have prepared him for his introduction to 
many of the scholastic activities he will encounter* 

Standard scores for the two levels of the tests are not equivalent* 
The standard scores used for the Spanish version are not strictly com- 
parable to English scores, since the difficulty levels of concepts 
according to language or culture have not been determined* 
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Staff Development 

Quantitative measures of this component are easily available, such 
as number of in-service meetings, teacher conferences and the like* In- 
stead of using these somewhat simplistic approaches, the design scheduled 
checks of movement toward designated competencies* Many of these com- 
petencies were observed by the curi’iculum specialist in individual inter- 
action with teachers and are somewhat difficult to quantify* 

In one instance, teachers were asked to assist in identifying 
common pronunciation errors in English as they are frequently made by 
Spanieh-speakipg kindergarten and first grade children. While a count 
of those named by a teacher or the whole group would be relatively 
meaningless, the fact that teachers worked to become more sensitive to 
developing language is likely to be of long term benefit to those 
teachers and, therefore, to bilingual education* 

In-service evaluation meetings were monitored by the evaluator or 
the testing specialist. It was recorded that, according to the design, 
sessions were held which related to pronunciation, first grade reading, 
and oral language development in both languages* Sessions in relating 
art, music and linguistics were also held for the staff. 

The materials indicated in the product objectives as dialogue 
practice have been developed as units and will be revised to be used as * 
resources for subsequent years* 

An observer report was developed for use with in-service meetings 
but its validity could not be practically established* It did aid the 
evaluator in reviewing the topics of in-service sessions but is deemed 
of insufficient value to retain. 
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An indirect measure of staff development was obtained through written 
responses of teachers and aides to questions put to them extemporaneously 
regarding their beliefs about what is meant by bilingual education. The 
responses were returned, some in fiiglish and some in Spanish and were re- 
viewed for apparent change of perceptions and comprehensiveness of thought. 
A few selected comments follow: 

From Aides 



Fall Soring 



La palfibra bilingue significa 
dominan 2 idomas. 

Bilingual education is trying 
to teach the child to live in 
an environment in which two 
languages are used. 

Bilingual education is a method 
by which we hope to teach our 
children to speak both English 
and Spanish fluently. 



Bilitrgual education means that the 
children who come to school without 
speaking English will have a better 
chance. It gives the teacher and 
aide a chance to work in the child’s 
native tongue and make him a better 
student for later days in school. 

Bilingual education means teaching 
the child in his native language 
a6 well as in English, or any other 
language. 

La educacion bilingue representa 
para me una cos a muy buena, ya que 
el nino se siente libre or cofiado 
de que puede expresar sus ideas en 
su idioma, sintiendose confiado de 
el, ya a la vez aprendiendo el idioma 
necesario el ingles. 



From Teachers 



Bilingual education is teaching the 
child (who functions in two languages) 
in those two languages. 

Bilingual education is the teaching 
and learning in two languages. The 
results would mean that the children 
should be proficient in the use of 
both languages— reading, writing, and 
speaking. 

Bilingual education is education 
using two languages to help the 
children to learn concepts. 



Teaching in both languages and 
teaching the child in the language 
in which he is most comfortable. 

Bilingual education is based on 
teaching the bilingual child in 
his native tongue and then pro- 
ceeding to teach him previously 
learned concepts in a second 
language. 
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While affective measures may be questioned, it would seem to be apparent 
from the foregoing representational responses that the staff continued 
to ’’value” bilingual education, which is among the higher ordered 
ratings of this domain. No statistical treatment of such data is in- 
dicated, but this does not negate its significance. 

Curriculum Development 

The staff was involved in day to day adaptation and trial of 
materials. In addition original units were constructed using various 
techniques and dialogues. A number of consultants assisted with spe- 
cific area development of curriculum and technique. A primary concern 
was the language experience approach to reading. No comparative group 
was utilised so evaluation is based upon achievement of criterion re- 
ferral items, which, at this point, do not lend themselves to sophisti- 
cated analysis. 

Teacher made tests, work inventories, and observations have been 
made regarding Spanish reading in first grade. Staff annotations are 
reflected in materials requests for 1971-72. It is the observation of 
the evaluator that the language experience approach in- Spanish reading 
has been more closely akin to sight comprehension approaches common to 
publishers of this country than it has to the single letter phonics 
approaches predominant in Mexico. Attenpt has been made to develop 
techniques which would be applicable in either language. 

Community Involvement 

As in previous instances, quantitative data is available for this 
component. Numbers of parents and others have attended orientation 

10 
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meetings, advisory meetings, school excursions and havo attended skill 
development (such as clothing repair) meetings held by community re- 
source people* The liason person has recorded frequent home visits* 

The quality of involvement is recognizably more difficult to 
assess* First hand observations and in-the-home interviews reveal a 
variety of perceptions of the bilingual program, almost all of which 
are rather strongly positive. The community acceptance of the program 
is also positively assessed indirectly from the high percent of at- 
tendance of kindergarten children* 

Observation forms were developed and used for a few meetings by 
the testing specialist* Since they were quite subjective, qualitative 
assessment of them seemed inappropriate* From more quantitative cri- 
teria, however, the involvement of rather large numbers of parents, in 
relation to parents of children in the same school but not in the bi- 
lingual program but who interact with the school, was noted* 

Sunmarv 

This narrative report of the evaluation of the Lubbock Bilingual 
Elementary Education Program for 1970-71 has concerned the internal, 
short term assessment of program components* In other terms it concerns 
the summative evaluation* There is little need to apply statistical 
analysis to the type of data obtained* 

The more lopg range, formative -type evaluation can also be drawn 
from the information contained herein without tests of statistical 
significance. There is evidence that a bilingual approach is in fact 
being' followed and that growth toward product and process objectives 
has been made. 
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A further statement by the evaluator, from professional training 
and sevor&l years experience in olomontary education i3 that tho program 
is functioning smoothly in the context of tho school ootting* There is 
ovidenco that constant monitoring has rosulted in program modification 
which is tho real measure of evaluation* 

It is still too early in tho program to attempt to specify major 
differences in the day-to-day behavior of those children from what 
might have been if they had not been afforded the bilingual schooling* 
Tho present evaluation frame, without a control group, was consciously 
selected becauso of tho difficulties attendant to designation of such 
controls* It is perhaps important to noto that this program is based 
upon achievement of educational objectives rathor than upon comparison 
of rate of gain* The measures used, even though tho validity of norm- 
referenced measures for young childron is doubtful and tho testing of 
young subjects is relatively unstable, do indicato that the objectives 
of the program are being approached* 




Len Ainsworth 
Evaluation Consultant 



